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EVALUATING  THE  RELATIONSHIP  BETWEEN  TEAM 
PERFORMANCE  AND  JOINT  ATTENTION  WITH  LONGITUDINAL 
MULTIVARIATE  MIXED  MODELS 

Michael  T.  Tolston1,  Adam  J.  Strang2,  Gregory  J.  Funke2,  Brent  Miller2,  Rebecca  Brown3,  Lauren  Menke3 
'Oak  Ridge  Institute  for  Science  and  Education,  Wright-Patterson  AFB,  OH;  2Air  Force  Research 
Laboratory,  Wright-Patterson  AFB,  OH;  3Ball  Aerospace  and  Technologies  Corporation,  Dayton,  Ohio 

Previous  research  indicates  that  measures  of  joint  attention  provide  unique  insight  into  team  cognition  and 
performance.  In  this  study,  we  examined  the  effects  of  practice  and  joint  attention  on  team  performance 
improvement  using  multivariate  mixed  models,  with  an  emphasis  on  exploring  the  correlation  structure 
between  the  variances  in  growth  trajectories  of  team  performance  and  joint  attention  around  estimated 
means.  Observed  patterns  in  team  performance  showed  time  dependent  trends  well  known  in  a  variety  of 
learning  contexts,  including  non-linear  growth,  performance  retention,  and  performance  retention  loss 
between  multiple  practice  sessions.  Joint  attention  was  found  to  decrease  non-linearly  over  time,  starting 
out  comparatively  high  and  decreasing  as  a  function  of  time  on  task.  Additionally,  we  found  evidence  of 
negative  relationships  between  joint  attention  and  team  performance  in  our  task  environment,  and  we 
established  that  joint  attention  was  significantly  higher  than  the  chance  levels  that  would  be  expected  by 
task-constraints  alone. 


Visual  gaze  patterns,  both  within  individuals  and  pairs, 
offer  unique  insight  into  the  cognitive  processes  that  help 
shape  behavior.  For  instance,  there  is  evidence  that  measures 
of  joint  attention  capture  underlying  similarities  in  the 
cognitive  processes  of  interacting  pairs  (Richardson,  Dale,  & 
Tomlinson,  2009),  and  may  also  index  the  establishment  of  a 
common  ground  understanding  that  helps  support  successful 
interpersonal  coordination  and  communication  (Vinson,  Dale, 
Tabatabaeian,  &  Duran,  2015).  In  applied  settings,  gaze 
patterns  have  been  shown  to  be  indicative  of  control  room 
operators’  readiness  and  capability  to  respond  to  abnormal 
situations  (Kodappully,  Srinivasan,  &  Srinivasan,  2016),  and 
are  related  to  the  expertise  level  of  surgeons  (Hermens,  Flin, 

&  Ahmed,  2013).  These  results  suggest  that  the  use  of  eye 
tracking,  as  a  means  to  record  gaze  patterns  that  serve  as 
proxies  for  attention,  may  provide  a  uniquely  sensitive  and 
non-invasive  method  for  indexing  the  evolution  of  team  states, 
readiness,  and  performance.  As  such,  understanding  the 
relationship  between  team  gaze  coordination  and  team 
performance  over  an  interval  in  which  learning  and  adaptation 
occur  in  a  complex  task  environment  can  provide  guidance  for 
the  development  of  team  diagnostic  procedures  and  training 
applications  (Hermens  et  al.,  2013).  However,  evaluating  the 
evolution  of  these  relationships  is  a  complicated  problem  that 
is  compounded  by  the  multiple  variables  associated  with 
complex  learning  (Van  Merrienboer  &  Sweller,  2005). 
Multivariate  analyses  using  mixed  models  offers  a  way  to 
approach  this  problem. 

Mixed  models  are  flexible  tools  for  analyzing  longitudinal 
relationships  between  variables,  as  they  are  well  suited  to 
handle  correlations  between  observations.  Multivariate  mixed 
models  are  an  extension  of  these  well-known  procedures  and 
can  be  approached  using  several  methods.  For  example,  a 
conditional  approach  models  one  outcome  variable  as  a 
function  of  a  time-varying  covariate.  This  method  has  several 
limitations,  including  the  possibility  of  misspecification 
(Curran,  Obeidat,  &  Losardo,  2010)  and  the  fact  that 


parameters  estimated  by  the  model  will  be  a  function  of  which 
covariate  is  entered  as  the  dependent  variable  -  the  latter  of 
which  may  confuse  interpretation  of  the  marginal  growth  of 
the  two  covariates  (Codd,  2014).  Another  approach  is  to 
model  multiple  dependent  variables  simultaneously,  and 
evaluate  their  dependence  by  allowing  within  participant 
variations  around  estimated  mean  values  to  be  correlated.  This 
yields  the  advantage  of  providing  estimations  of  the  changes 
within  each  variable,  as  well  as  their  mutual  relationships. 

The  goal  for  this  study  was  to  employ  a  multivariate 
mixed  models  approach  to  identify  changes  in  patterns  of  joint 
attention  associated  with  task  performance  in  a  longitudinal 
design.  We  employed  a  multivariate-modeling  framework  to 
investigate  the  temporal  evolution  of  team  performance  in  a 
medium-fidelity  simulated  task  environment  and  the 
associated  changes  in  joint  attention  that  are  posited  to  capture 
similar  or  shared  cognitive  states.  In  as  much  as  the  task 
requires  coordinated  activity  and  shared  understanding  of  task 
constraints,  we  predicted  that  higher  magnitudes  of  joint 
attention  would  be  positively  related  to  successful  task 
outcomes.  Additionally,  we  sought  to  investigate  the  influence 
of  pre-existing  social  ties  on  joint  attention  and  performance 
by  evaluating  differences  between  teams  comprised  of  friends 
and  teams  with  no  prior  relationship. 

METHODS 

Participants 

Forty-eight  participants  took  part  in  this  study.  Due  to 
incomplete  data,  three  teams  were  omitted  from  analyses. 

Ages  in  the  remaining  21  same-sex  dyads  (16  women  and  26 
men)  ranged  from  18  to  35  (M  =  23.98,  SD  =  4.33).  Of  these, 
ten  were  comprised  of  self -identified  friends,  while  the  other 
1 1  were  composed  of  pairs  who  had  not  met  before. 

Design 

The  experiment  was  a  2  (friends)  x  3  (session)  x  12  (trial) 
x  2  (task  difficulty)  mixed  design,  where  friends  was  a 
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between-groups  factor,  and  session,  trial,  and  task-difficulty 
were  within-groups  factors.  For  this  report,  we  focused  on  the 
changes  in  joint  attention  and  performance  during  the  first  two 
sessions,  where  task  difficulty  remained  constant,  yielding  a  2 
(friend)  x  2  (session)  x  12  (trial)  mixed  design. 

Materials  and  Apparatus 

Gaze  data  were  collected  using  two  desk-mounted  Seeing 
Machines  Inc.,  faceLAB  eye  trackers  recording  at  60  Hz. 
Participants  sat  at  desks  in  padded  chairs  95  cm  from  separate 
Samsung  Syncmaster  2443  60.96  cm  LCD  monitors  with 
1280x1024  displays. 

Team  Task.  Participants  engaged  in  the  Experimental 
Research  Environment  for  Supervisory  Control  of 
Heterogeneous  Unmanned  (RESCHU)  Vehicles  synthetic  task 
environment  (see  Figure  1).  In  RESCHU,  participants  work 
together  to  control  several  unmanned  aerial  vehicles  (UAVs) 
with  the  ultimate  goal  of  locating  specific  objects  of  interest  in 
urban  coastal  and  inland  settings.  Participants  had  distinct 
roles  in  the  simulation.  The  navigator  was  able  to  see  the 
entire  map,  including  all  UAVs,  all  targets,  and  all  threats;  the 
pilot  controlled  the  UAVs  via  a  point  and  click  interface,  but 
was  only  able  to  see  targets  and  threats  in  the  sectors 
populated  by  UAVs.  Thus,  for  the  team  to  approach  a  target 
successfully  with  a  UAV,  the  navigator  would  use  the  chat 
interface  to  indicate  to  the  pilot  the  location  of  the  target  and 
any  threats  along  the  path  from  a  chosen  UAV  to  the  specified 
target.  After  a  UAV  arrived  at  an  area  of  interest,  either 
participant  could  engage  in  a  visual  search  task.  This 
secondary  task  consisted  of  looking  for  a  particular  target  in 
an  image  by  panning  and  zooming  a  simulated  camera  feed. 
Visual  search  and  control  of  UAVs  could  be  conducted  in 
parallel.  The  goal  of  the  RESCHU  task  was  for  participants  to 
maximize  their  performance  by  1)  reaching  as  many  areas  of 
interest  as  possible,  2)  correctly  completing  as  many  of  the 
target  detection  tasks  as  possible,  and  3)  avoiding  threat  areas. 
Performance  on  this  task  was  measured  by  the  total  number  of 
arrivals  at  targets  plus  the  total  number  of  successful  search 
tasks,  minus  the  number  of  threat  areas  encountered  in  each 
trial. 

Procedure 

Written  informed  consents  were  obtained  from  all 
participants  upon  arrival  for  the  first  session.  Participants  were 
then  given  instructions  regarding  the  RESCHU  task  and  one 
participant  from  each  dyad  was  assigned  at  random  to  the  role 
of  navigator,  while  the  other  was  assigned  to  be  the  pilot. 
Participants  maintained  their  assigned  roles  throughout  the 
experiment.  During  the  study,  participants  sat  at  separate 
desks  situated  along  a  common  wall  with  3.05  m  between 
them.  Participants  were  equipped  with  physiological 
monitoring  sensors,  including  heart  rate, 
electroencephalography  (data  from  these  sensors  will  be 
reported  elsewhere),  and  the  eye-tracking  equipment  described 
earlier  (calibrated  to  less  than  2  degrees  of  error  in  measured 
visual  angle).  To  prevent  interference  with  physiological 
monitoring  equipment,  participants  were  asked  not  to  talk  to 
one  another,  and  to  communicate  only  via  the  RESCHU  chat 


interface.  Teams  took  part  in  12  ten-minute  trials  during  each 
session.  Teams  received  a  ten-minute  break  every  three  trials, 
with  an  optional  half-hour  break  between  the  sixth  and  seventh 
trials.  Each  session  lasted  approximately  eight  hours,  with  the 
two  sessions  separated  by  three  to  five  days,  dictated  by 
participant  scheduling  constraints. 

Analyses  and  Models 

Gaze.  Gaze  data  were  pre-processed  by  removing  samples 
for  which  an  eye  tracking  system  lost  track  of  both  the  pupils 
and  irises  of  either  participant.  Gaze  locations  greater  than  100 
pixels  outside  of  the  calibration  area  were  also  removed. 
Sections  of  data  in  which  there  were  greater  than  250  ms  of 
contiguously  missing  data  for  either  participant  in  a  pair  were 
deemed  bad,  and  files  that  contained  greater  than  10%  bad 
data  were  removed  from  subsequent  analyses  (resulting  in  the 
removal  of  two  files).  A  cubic  spline  interpolation  was  used  to 
replace  removed  samples,  and  data  were  then  down-sampled 
to  20  Hz. 


Figure  1.  The  RESCHU  interface.  Participants  worked  together  to  navigate 
UAVs  (green  icons)  to  targets  (diamonds)  while  avoiding  threat  areas  (yellow 
circles).  A  visual  search  task  appeared  in  the  upper  left  corner  upon  UAV 
arrival  at  a  target.  The  left  center  window  is  a  chat  interface,  and  the  left 
bottom  window  contains  indicators  of  UAV  engagement. 

Joint  attention  was  estimated  from  the  separate  two- 
dimensional  time  series  of  screen  coordinates  (in  pixels)  of 
team  gaze  data  using  Cross  Recurrence  Quantification 
Analysis  (CRQA)  (Marwan,  Carmen  Romano,  Thiel,  & 
Kurths,  2007;  Richardson,  Dale,  &  Kirkham,  2007; 
Richardson  et  ah,  2009).  This  technique  quantifies  the  degree 
of  similarity  between  two  different  sets  of  time  series.  With 
respect  to  the  parameters,  no  delay  embedding  was  used,  the 
radius  was  set  to  250  pixels,  and  data  were  analyzed  within  a 
30-second  window  from  the  diagonal.  This  means  that  when 
participants  were  looking  at  locations  of  their  screens  within 
250  pixels  of  each  other,  and  within  the  30  s  window,  that 
sample  was  identified  as  recurrent.  Percent  recurrence 
(%REC)  was  then  calculated  as  the  number  of  observed 
recurrences  to  the  total  number  of  possible  recurrences  for 
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each  10  minute  trial.  Simply  put,  %REC  with  these  parameter 
settings  gives  the  percentage  of  time  participants  spent  looking 
at  the  same  areas  of  their  displays  within  30  s  of  each  other.  It 
should  be  noted  that  prior  work  has  used  smaller  windows  to 
evaluate  joint  attention  (e.g.,  3  s  in  Richardson  et.  al,  2009). 
However,  in  our  preliminary  analyses,  we  found  qualitatively 
similar  results  for  %REC  between  3  s  (M  =  16.64,  SO  =  1.19) 
and  30  s  (M  =  16.03,  SD  =0.97)  windows  over  the  10  minute 
trial.  We  used  the  larger  30  s  window  so  that  in  a  future 
publication  we  might  be  able  to  compare  joint  attention  with 
other  measures  that  require  larger  amounts  of  data.  As  a  test  of 
joint  attention  versus  task-constrained  gaze  patterns,  team 
%REC  was  compared  against  surrogate  %REC  values  that 
were  calculated  from  virtual  pairings  of  navigators  and  pilots 
from  30  different  groups  (Shockley,  Baker,  Richardson,  & 
Fowler,  2007;  Strang,  Funke,  Sheldon,  Dukes,  &  Middendorf., 
2014). 

Models.  Linear  mixed  model  analyses  were  conducted  in 
R  (R  Core  Team,  2015)  using  the  package  nlme  (Pinheiro, 
Bates,  Sarkar,  &  R  Core  Team,  2015).  To  prevent  spurious 
correlations  between  slopes  and  intercepts,  the  time  variable 
was  centered,  meaning  that  all  intercepts  are  to  be  interpreted 
as  projected  average  values  for  the  time  between  trials  12  and 
13.  To  allow  comparison  of  nested  models,  the  Maximum 
Likelihood  estimator  was  used.  A  sequential  analysis  method 
was  used  in  which  the  best  univariate  model  was  found  for  the 
individual  variables,  which  were  then  combined  in  a 
multivariate  model  to  test  for  relationships  between  random 
effects  (MacCallum,  Kim,  Malarkey,  &  Kiecolt-Glaser,  1997). 
The  following  terms  were  entered  sequentially,  first  as  fixed 
effects  and  then  as  random  effects,  starting  from  the  null 
model  fitting  only  the  intercept  with  a  random  term  added  for 
group:  trial  (time),  trial2,  session,  and  friends.  For  clarification, 
the  trial2  allows  the  slope  to  vary  as  the  second  order  function 
of  trial,  which  captures  quadratic  trends  in  the  data.  Each  term 
was  also  tested  for  group  level  random  effects.  The  most 
parsimonious,  best  fitting  model  for  each  variable  was 
determined  from  likelihood  ratio  tests  of  nested  models. 

Visual  inspection  of  the  normalized  Quartile-Quartile  plots  of 
the  model  residuals  versus  observed  values  revealed  three 
points  that  were  identified  as  possible  outliers.  Removal  of 
these  values  did  not  significantly  alter  the  fit  of  either  model. 
Reported  results  are  for  the  multivariate  model  estimated  with 
these  data  points  removed. 

The  univariate  models  were  combined  and  analyzed  using 
the  techniques  specified  in  MacCallum  et  al.  (1997),  in  which 
each  of  the  dependent  variables  are  stacked  into  a  single 
column  vector  with  two  dummy  column  vectors  specifying  the 
contribution  of  each  in  the  model.  The  random  effects 
variance-covariance  matrix  for  the  null  model  had  a  mean  of 
zero,  with  full  specification  of  covariances  within  performance 
and  %REC,  and  with  zero  covariances  between  random  effects 
estimated  for  performance  and  %REC.  As  the  two  variables 
were  measured  on  different  scales,  separate  residual  variables 
were  required  in  the  multivariate  model.  We  thus  specified 
zero  mean  residuals  with  independent  variance  components 
for  both  performance  and  %REC.  The  multivariate  model  was 
specified  as: 


y irk*  ^ Scor\  P ))U  score)  ^  P\i)  scoreY^it  Pli{score)^it  fi'Siisorres^i) 

+  ei(ra>re)]  +  <^%REc[A)i(%«EC)  +  Pli(%REC)Xit  +  PlH%REC)Xit  +  en%REC)]’ 


(1) 


where  y /,*-*  is  a  measure  for  team  i  at  time  t  for  variable  k*\ 
Sscore  and  6%rec  are  dummy  variables  for  team  performance 
(score)  and  %REC.  Time  for  team  i  at  trial  t  is  captured  by  xu, 
while  the  quadratic  effect  of  time  is  captured  by  x\.  The  effect 
the  fh  session  for  team  i  is  captured  by  x,y  The  /?  coefficients 
were  specified  to  capture  fixed  means  for  all  variables, 
random  effects  for  the  intercepts  of  performance  and  %REC, 
the  first  and  second  order  effects  of  trial  on  performance,  the 
first  order  effect  of  trial  on  %REC,  and  random  effects  for 
these  later  components.  As  a  final  test  for  dependency 
between  growth  trajectories  of  %REC  and  performance,  an 
alternative  model  with  the  same  form  as  (1)  was  constructed, 
but  the  constraints  forcing  non-interacting  variances  and 
covariances  of  random  effects  between  performance  and 
%REC  were  relaxed. 


RESULTS 

Plots  for  performance,  %REC,  and  %REC  surrogate 
analyses  as  a  function  of  trial  can  be  seen  in  Figure  2.  There  is 
an  obvious  upward  trend  for  performance,  with  a  decreasing 
slope  as  a  function  of  trial.  Similar  observations  can  be  made 
of  %REC,  though  the  trends  are  in  the  opposite  directions.  A 
single  sample  r-test  comparing  the  average  %REC  for  each 
team  against  the  average  surrogate  %REC  shows  that  %REC 
for  actual  teams  was  higher  than  what  would  be  expected  by 
chance  alone,  t( 20)=  4.88,  p  <  .001,  though  this  difference 
lessens  over  time. 
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Figure  2.  Mean  values  of  %REC  of  team  gaze  data,  %REC  of  surrogate  gaze 
data,  and  team  performance  plotted  against  trial.  Error  bars  represent  ±1 
standard  error  of  the  mean. 
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The  final  multivariate  model  allowing  covariances 
between  random  effects  estimated  for  %REC  and  performance 
resulted  in  a  significantly  better  fit  compared  to  the  null 
model,  -2LLx2  (6)  =  17.48,  p  =  .001,  meaning  that  there  were 
significant  relationships  between  the  growth  trajectories  of 
%REC  and  performance  not  accounted  for  by  the  intravariable 
fixed  and  random  effects.  Parameter  estimates  from  the  final 
multivariate  model  can  be  seen  in  Tables  1  and  2,  while  plots 
of  the  estimated  trajectories  for  each  pair  can  be  seen  in  Figure 
3.  The  significant  intercepts  for  performance  and  %REC 
represent  the  projected  average  values  for  the  time  between 
trials  12  and  13.  The  significant  linear  slope  for  performance 
shows  that,  as  trials  progressed,  teams  became  better  at  the 
task,  though  the  quadratic  change  in  slope  in  the  negative 
direction  indicates  that  this  trend  was  decreasing  over  time, 
likley  capturing  an  asymptotic  approach  to  some  ceiling  or 
plateau  in  peformance.  The  fixed  effect  of  session  on 
performance  means  that  the  observed  values  in  session  two 
were  lower  than  they  would  have  been  if  the  trends  from 
session  one  continued  without  interruption.  The  significant 
negative  slope  for  %REC  shows  that,  as  trials  progressed,  the 
joint  attention  within  pairs  decreased,  though  the  positive 
second  order  change  in  slope  shows  that  the  magnitude  of  this 
negative  trend  decreased  over  time.  In  Figure  1,  it  can  be  seen 
that  the  mean  value  of  %REC  was  approaching  chance  levels, 
possibly  indicating  that  participants  were  adopting 
increasingly  independent  gaze  behaviors  as  time  progressed. 


Trial 

Figure  3.  Estimated  values  of  %REC  and  performance  for  individual  teams, 
demonstrating  the  overall  growth  trajectories  captured  by  the  fixed  effects,  as 
well  as  the  within  group  variability  modeled  by  the  random  effects.  Broken 
lines  correspond  to  the  trials  that  were  removed  from  analysis  (either  due  to 
missing  data  or  from  being  flagged  as  possible  outliers). 

We  were  particularly  interested  in  the  variation  and 
covariation  of  random  effects  of  the  individual  pairs  around 


estimated  mean  intercepts  and  slopes.  The  negative  correlation 
between  the  intercept  of  performance  and  %REC  indicates 
that  an  increased  amount  of  joint  attention  above  the  overall 
mean  was  associated  with  poorer  performance,  while  the 
positive  relation  between  the  slope  of  %REC  and  performance 
indicates  that  groups  with  higher  %REC  slopes  tended  to  have 
better  performance  by  the  end  of  session  one  than  groups  that 
had  lower  slopes  of  %REC.  This  seems  to  somewhat 
contradict  the  finding  that  associates  higher  %REC  with  lower 
performance,  though  it  may  be  explained  by  positing  that 
groups  who  started  out  with  relatively  low  %REC,  and  thus 
were  likely  to  have  had  smaller  negative  slopes,  peformed 
better  than  groups  that  started  with  high  %REC,  and  thus  were 
likely  to  have  had  larger  negative  slopes.  This  is  supported  by 
the  negative  relationship  between  the  %REC  intercept  and 
slope. 


Table  1. 

Estimates  of  Fixed  Effects  for  Multivariate  Model 


Coefficient 

Estimate 

SE 

t 

P 

Performance  Intercept 

56.435 

2.538 

22.238 

<.001 

%REC  Intercept 

15.654 

0.532 

29.422 

<.001 

Performance  x  Trial 

1.672 

0.094 

17.852 

<.001 

Performance  x  Trial2 

-0.049 

0.009 

-5.402 

<.001 

Performance  x  Session 

-6.307 

1.109 

5.686 

<.001 

%REC  x  Trial 

-0.117 

0.026 

-4.445 

<.001 

%REC  x  Trial 2 

0.007 

0.002 

3.206 

0.001 

Note.  The  fixed  effects  of  Performance  and  %REC  intercepts  indicate  the 
projected  intercepts  between  sessions  12  and  13 


Table  2. 

Estimates  of  Random  Effects  for  Multivariate  Model  of  Change  in  %REC 
and  Performance 


Variances,  Covariances,  and 
Intracorrelations  of  Random  Effects 


Perf. 

%REC 

Perf. 

Perf. 

%REC 

Intercept 

Intercept 

X 

Trial 

X 

Trial2 

X 

Trial 

Perf. 

47.641 

-8.660 

0.428 

-0.040 

0.523 

%REC 

-0.326 

9.657 

-0.965 

0.048 

-0.243 

Perf.  X  Trial 

0.138 

-0.005 

0.577 

-0.022 

0.025 

Perf.  X  Trial2 

-0.684 

0.404 

-0.224 

0.001 

-0.002 

%REC  X  Trial 

0.680 

-0.521 

-0.537 

-0.545 

0.010 

Residual  Variances: 

Performance 

38.025 

%REC 

4.592 

Note.  The  numbers  in  the  upper  right  triangle  and  along  the  diagonal  are 
variances  and  covariances,  while  the  numbers  in  the  lower  left  triangle  are 
correlations.  Perf.  =  Performance. 


DISCUSSION 

In  this  study,  we  evaluated  the  growth  curves  of  team  task 
performance  and  joint  attention  in  a  complex  team-task 
environment,  with  special  emphasis  on  the  multivariate 
analysis  of  the  relationship  between  variations  in  individual 
group  performance  and  joint  attention  around  estimated 
means.  We  found  well-known  characteristics  of  task  learning 
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in  the  performance  variable,  which  increased  over  time  with  a 
decreasingly  positive  slope,  meaning  that  teams  performed 
asymptotically  better  as  trials  progressed.  There  was  also  a 
negative  estimated  fixed  effect  of  session,  indicating  a 
performance  retention  loss  between  sessions,  but  the  small 
magnitude  of  the  decrement  indicated  an  overall  net  retention 
of  learned  behavior.  We  found  a  similar  curve  in  joint 
attention,  though  in  the  opposite  direction,  and  without  the 
effect  of  session.  On  average,  teams  started  out  with  a  higher 
degree  of  joint  attention,  and  this  value  decreased 
asymptotically  over  time.  With  respect  to  the  relationship 
between  performance  and  joint  attention,  we  found  that  lower 
overall  levels  of  joint  attention  were  positively  correlated  with 
better  performance,  while  the  slope  of  joint  attention  was 
positively  correlated  with  better  performance.  These  latter 
findings  indicate  that  teams  who  started  out  with  lower 
magnitudes  of  joint  attention  decreased  less  in  this  measure 
over  time,  which  is  borne  out  in  the  negative  correlation 
between  %REC  intercept  and  its  slope.  Finally,  we  found  no 
effects  of  friendship  on  either  joint  attention  or  team 
performance,  which  indicates  that  pre-established  social  ties 
do  not  necessarily  lead  to  better  performance  in  a  cooperative 
task. 

Our  findings  with  respect  to  the  qualitative  change  in 
performance  are  consistent  with  prior  studies  of  learning, 
retention,  and  retention  loss  (Adams,  1987).  In  that  our  task 
required  coordination  and  communication  between  teammates, 
we  expected  a  positive  relationship  between  task  performance 
and  %REC,  but  found  instead  a  negative  relation.  We  note, 
however,  that  our  present  findings  are  in  line  with  those  of 
Strang  and  colleagues  (2014),  who  reported  a  negative 
relationship  between  physio-behavioral  coupling  and  task 
performance  in  teams  playing  a  cooperative  video  game.  The 
authors  hypothesized  that  efficient  team  strategies  might 
emphasize  unique  contributions  of  each  individual,  which 
would  decrease  the  overall  amount  of  similar  physio- 
behavioral  states.  It  is  possible  that  the  larger  magnitudes  of 
joint  attention  observed  in  the  earlier  stage  of  our  present 
experiment  indicated  a  learning  phase,  in  which  participants 
established  a  common  ground  understanding  of  the  task 
constraints.  However,  as  the  task  progressed  and  this  common 
understanding  helped  establish  effective  and  efficient 
communication  patterns,  participants  were  increasingly  free  to 
perform  independent  roles,  possibly  with  the  navigator 
spending  less  time  looking  at  the  UAV  map,  and  more  time 
engaging  in  the  visual  search  task.  Further  investigations  into 
these  hypotheses  are  ongoing. 

We  did  not  expect  the  lack  of  influence  of  friendship  on 
performance  or  joint  attention,  as  prior  work  has  demonstrated 
that  friendship  can  facilitate  performance  in  decision-making 
and  motor  tasks  (e.g.,  Shah  &  Jehn,  1993).  However,  a  recent 
meta-analytic  review  of  the  effect  of  friendship  on  team 
performance  indicates  that  the  positive  relationship  is 
mediated  by  the  degree  of  task  interdependence  (Chung, 

2015).  This  reinforces  our  interpretation  that  role 
differentiation  played  a  key  part  in  the  development  of  team 
performance  over  time. 


Fimitations  of  this  study  include  that  the  task  specific 
constraints  likely  limit  the  generalization  of  our  findings 
across  team  tasks  that  demand  continuous,  uninterrupted 
collaboration.  In  this  task,  once  directions  are  given  to  the 
pilot  by  the  navigator,  the  navigator  is  free  to  engage  in  the 
visual  search  task  while  the  pilot  controls  the  UAVs.  Tasks  in 
which  such  behavioral  independence  is  not  possible  are  likely 
to  show  relationships  between  joint  attention  and  performance 
different  from  those  observed  in  the  present  study. 

Overall,  the  results  of  this  experiment  provide  evidence 
supporting  the  use  of  joint  attention  and  multivariate  mixed 
models  in  understanding  the  evolution  of  team  performance  in 
complex,  coordinative  task  environments. 
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